Edge Guided GANs with Multi-Scale Contrastive Learning for Semantic Image Synthesis

We propose a novel ECGAN for the challenging semantic image synthesis task. Although considerable improvements have been achieved by the community in the recent period, the quality of synthesized images is far from satisfactory due to three largely unresolved challenges. 1) The semantic labels do not provide detailed structural information, making it challenging to synthesize local details and structures; 2) The widely adopted CNN operations such as convolution, down-sampling, and normalization usually cause spatial resolution loss and thus cannot fully preserve the original semantic information, leading to semantically inconsistent results (e.g., missing small objects); 3) Existing semantic image synthesis methods focus on modeling 'local' semantic information from a single input semantic layout. However, they ignore 'global' semantic information of multiple input semantic layouts, i.e., semantic cross-relations between pixels across different input layouts. To tackle 1), we propose to use the edge as an intermediate representation which is further adopted to guide image generation via a proposed attention guided edge transfer module. To tackle 2), we design an effective module to selectively highlight class-dependent feature maps according to the original semantic layout to preserve the semantic information. To tackle 3), inspired by current methods in contrastive learning, we propose a novel contrastive learning method, which aims to enforce pixel embeddings belonging to the same semantic class to generate more similar image content than those from different classes. We further propose a novel multi-scale contrastive learning method that aims to push same-class features from different scales closer together being able to capture more semantic relations by explicitly exploring the structures of labeled pixels from multiple input semantic layouts from different scales.Comment: Accepted to TPAMI, an extended version of a paper published in ICLR2023. arXiv admin note: substantial text overlap with arXiv:2003.1389

Fine-grained Recognition: Accounting for Subtle Differences between Similar Classes

The main requisite for fine-grained recognition task is to focus on subtle discriminative details that make the subordinate classes different from each other. We note that existing methods implicitly address this requirement and leave it to a data-driven pipeline to figure out what makes a subordinate class different from the others. This results in two major limitations: First, the network focuses on the most obvious distinctions between classes and overlooks more subtle inter-class variations. Second, the chance of misclassifying a given sample in any of the negative classes is considered equal, while in fact, confusions generally occur among only the most similar classes. Here, we propose to explicitly force the network to find the subtle differences among closely related classes. In this pursuit, we introduce two key novelties that can be easily plugged into existing end-to-end deep learning pipelines. On one hand, we introduce diversification block which masks the most salient features for an input to force the network to use more subtle cues for its correct classification. Concurrently, we introduce a gradient-boosting loss function that focuses only on the confusing classes for each sample and therefore moves swiftly along the direction on the loss surface that seeks to resolve these ambiguities. The synergy between these two blocks helps the network to learn more effective feature representations. Comprehensive experiments are performed on five challenging datasets. Our approach outperforms existing methods using similar experimental setting on all five datasets.Comment: To appear in AAAI 202

CompositeTasking: Understanding Images by Spatial Composition of Tasks

We define the concept of CompositeTasking as the fusion of multiple, spatially distributed tasks, for various aspects of image understanding. Learning to perform spatially distributed tasks is motivated by the frequent availability of only sparse labels across tasks, and the desire for a compact multi-tasking network. To facilitate CompositeTasking, we introduce a novel task conditioning model -- a single encoder-decoder network that performs multiple, spatially varying tasks at once. The proposed network takes an image and a set of pixel-wise dense task requests as inputs, and performs the requested prediction task for each pixel. Moreover, we also learn the composition of tasks that needs to be performed according to some CompositeTasking rules, which includes the decision of where to apply which task. It not only offers us a compact network for multi-tasking, but also allows for task-editing. Another strength of the proposed method is demonstrated by only having to supply sparse supervision per task. The obtained results are on par with our baselines that use dense supervision and a multi-headed multi-tasking design. The source code will be made publicly available at www.github.com/nikola3794/composite-tasking

Genomic evidence of adaptive evolution in the reptilian SOCS gene family

The suppressor of the cytokine signaling (SOCS) family of proteins play an essential role in inhibiting cytokine receptor signaling by regulating immune signal pathways. Although SOCS gene functions have been examined extensively, no comprehensive study has been performed on this gene family’s molecular evolution in reptiles. In this study, we identified eight canonical SOCS genes using recently-published reptilian genomes. We used phylogenetic analysis to determine that the SOCS genes had highly conserved evolutionary dynamics that we classified into two types. We identified positive SOCS4 selection signals in whole reptile lineages and SOCS2 selection signals in the crocodilian lineage. Selective pressure analyses using the branch model and Z-test revealed that these genes were under different negative selection pressures compared to reptile lineages. We also concluded that the nature of selection pressure varies across different reptile lineages on SOCS3, and the crocodilian lineage has experienced rapid evolution. Our results may provide a theoretical foundation for further analyses of reptilian SOCS genes’ functional and molecular mechanisms, as well as their roles in reptile growth and development

Evaluation of an identification method for the SARS-CoV-2 Delta variant based on the amplification-refractory mutation system

The Delta variant of SARS-CoV-2 dominated the COVID-19 pandemic due to its high viral replication capacity and immune evasion, causing massive outbreaks of cases, hospitalizations, and deaths. Currently, variant identification is performed mainly by sequencing. However, the high requirements for equipment and operators as well as its high cost have limited its application in underdeveloped regions. To achieve an economical and rapid method of variant identification suitable for undeveloped areas, we applied an amplification-refractory mutation system (ARMS) based on PCR for the detection of novel coronavirus variants. The results showed that this method could be finished in 90 min and detect as few as 500 copies/mL and not react with SARS-Coronavirus, influenza A H1N1(2009), and other cross-pathogens or be influenced by fresh human blood, α- interferon, and other interfering substances. In a set of double-blind trials, tests of 262 samples obtained from patients confirmed with Delta variant infection revealed that our method was able to accurately identify the Delta variant with high sensitivity and specificity. In conclusion, the ARMS-PCR method applied in Delta variant identification is rapid, sensitive, specific, economical, and suitable for undeveloped areas. In our future study, ARMS-PCR will be further applied in the identification of other variants, such as Omicron

Coordinated Relationship between Compactness and Land-Use Efficiency in Shrinking Cities: A Case Study of Northeast China

Compact development and efficient land use are effective ways to address the development dilemma and boost the vitality of shrinking cities. Moreover, it is critical to investigate the relationship between compactness and land-use efficiency in order to healthily and sustainably develop shrinking cities. This study developed an analytical framework to study the coordinated interaction between urban compactness and land-use efficiency in the context of city shrinkage. Fifteen typical shrinking cities in Northeast China were taken as an example of the phenomenon and the entropy value method and super-slack-based measure model were used to quantitatively measure the compactness and land-use efficiency. Furthermore, the coordinated development level and the coordinated relationship between the two were explored with the help of the coupled coordination degree model as well as the quadrant diagram method. The results of the study show that: (1) The overall level of compactness of 15 shrinking cities in Northeast China was low and the improvement of land-use efficiency was not obvious, while the differences of compactness and land-use efficiency between shrinking cities were significant. (2) The coordination between compactness and land-use efficiency was limited; however, the overall coordination remained intermediate, with significant spatial differences and a tendency to further expand. The problem of lagged development of land-use efficiency in shrinking cities could be characterized as serious. (3) Economic development and population size promote the coordinated development of urban compactness and land-use efficiency, whereas science and technology development, industrial structure, as well as government regulation inhibited the coordinated development of urban compactness and land-use efficiency